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FOEEWOKD 


The Second Hational Sjaaposiiom on Computerized Structural Analysis 
and Design was held on the campus of The George Washington University 
on March 29-31, 1976. The purpose of the Second Symposium was to 
facilitate the exchange of information among structural analysts and design 
and research engineers on recent developments in structural engineering 
and applied mechanics with special emphasis on the utilization of com- 
puters in the analysis and design of structures and their components. 

As a part of the symposium, a panel discussion was held on Standardi- 
zation, Certification, Maintenance, and Dissemination of Large Scale 
Engineering Software Systems. The panel moderator and panel members 
are listed on the following page. The panel discussion consisted of 
two basic parts: 

PART I - PRESENTATION BY PANEL MEMBERS: Each member of the panel presented 

a brief discussion on a specific topic associated with the theme 
of the panel discussion. 

PART II - AUDIENCE RESPONSE AND DISCUSSIONS: This consisted of a cross- 

discussion between the audience (symposium participants) and the 
panel members on topics that were either discussed earlier by the 
panel members or that were pertinent to the discussion. 



The discussions by panel members and the response by the audience were 
recorded on magnetic tape. In view of the interest shown by many individuals 
both during the meeting and subsequent to the symposium, a written documen- 
tation o£ the panel discussion was prepared although the proceedings of the 
symposium itself will not be published. To a great extent, this report 
represents a verbatim transcript of the presentations of the panel members 
and the responses of the audience. Mr. Mehmet I. Basci, Graduate Research 
Assistant at The George Washington University, helped in the drawing and 
preparation of the figures appearing in the publication. 
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PART I - PRESENTATION BY PANEL MEMBERS 


Harvey McComb> NASA Langley Research Center, Moderator : 

Good afternoon ladies and gentlemen. Welcome to our panel discussion on 
Standardization, Certification, Maintenance, and Dissemination of Large 
Engineering Software Systems. We are aware that the field of computerized 
structural analysis and design is growing at a very rapid rate. It has been 
described as chaotic and undisciplined. This situation is natural, I suppose, 
in a field so relatively new and where many bright, aggressive,, and innovative 
people are involved. It means that people trying to make effective use of 
computerized methods have many options available, but it also means that 
difficult choices must be made both in terms of software and hardware. Many 
people feel that it is important to bring some sort of order or discipline 
into the field if it can be done without stifling too much the rich variety 
and innovation. To try to get some handle on the subject, we have asked 
the people in the panel to address specific areas in their opening remarks. 
This breakdown is not clear but rather arbitrary and overlapping. Neverthe- 
less, perhaps, it will help focus our thoughts, I will introduce .he panel 
members now-. 

Starting on my right we have Stan Hansen, who is Manager of Flexstab 
System and Advanced System Development for Boeing Computer Services. I 
would like to ask Stan to address himself to Program Design and Development. 


Next is Harry White, who is Associate Director for ADP Standards of the 

\ 

Institute for Computer Sciences and Technology of the National Bureau of 
Standards. He will be talking about Standards and Documentation. 

Robert Nickell, Supervisor of the Design Technology Division of Nuclear 
Fuel Cycle Development Department at Sandia Laboratory. Bob will be dis- 
cussing Software Validation and Gertific \tion. 

Then on my left is James Johnson, Aerospace Engineer at the Air Force 
Flight Dynamics Laboratory. Jim will address himself to Dissemination, Porta- 
bility, and Maintenance. 

Next is Steve Fenves, who is Professor of Civil Engineering at Camegre- 
Mellon University, and I asked Steve to address the subject of Education. 

Then on the far left is Mike Gaus, Head of the Engineering Mechanics 
Section of the Engineering Division of the National Science Foundation. 

I want to give the panel members a chance to make opening remarks 
first. This will take us up to the coffee break. After break we will 
allow the panel to react to what they have heard from each other and 
make any other comments they wish. Then we will open up the discussion 
to members of the audience. 

We do hope to put out a publication which captures the essence of 
this discussion and we are trying to tape the discussion here. (Please 
keep that in mind when you make your comments.) In addition, if there 
are things later on, if you would like to send us a written digest of 
remarks or comments you would like to make, we will be glad to include 
this material in our published volume if it seems appropriate. 
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Ok, we will get started. ¥e will start off with. Stan Hansen on my right. 
Stan, as I mentioned, will be talking about Program Design and Development. 

Stan Hansen, Boeing Computer Services ; 

I should place myself in context, that is, I have -worked in the aerospace 
industries since the initiation .of my career in this field. We deal with very 
large systems. Therefore, what I am going to say will seem a little bit diffe- 
rent to some of you who are dealing with smaller systems. I keep figure 1 on 
my biilletin board. When I began my present assignment, the subsystem of which 
this subroutine is a part was being documented. This and several other flow- 
charts like it had been prepared for inclusion in the documentation. This flow- 
chart would be humorous if the process that produced it had not been so seriously 
deficient , 

This particular subsystem had been developed on an emergency basis. A pro- 
grammer had been brought in on loan from another organization, told there were 
no specifications, no time for design, that he was to take his direction from 
the other engineers and programmers, and to finish in three months. Nineteen 
months later, when I came on the project, the programmer was still working. He 
had generated 15,000 new coding statements in addition to the 10,000 that had 
existed. 

We managed to close out the work in another two months. Today, eighteen 
months later, we are finding that the code in this subsystem is so much more 
complex than the functions being computed, in many cases, it is more cost effec- 
tive for us to reprogram than to attempt to maintain. 

And this is the point: complexity. Engineering software systems are com- 

plex. They become unmanageable if the complexity is out of control, as illus- 
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trated by tba above example and by figure 2. Uncontrolled complexity is a 
major contributor to today’s software problems (fig. 3); 

Unsatisfied user requirements 
Unpredictable costs and schediiles 
Low technical quality 
Ftmctional unreliability 
Fragile software 
Difficult maintenance 

Complexity seems to be produced by three factors (fig. 4) ; 

The number of components interacting 

The number of interactions between components 

The complexity of the interactions 

These three factors apply whether the components are individuals, organi- 
zations, engineering specifications, software design, code, etc. If the software 
system and the organizations and functions which produce and maintain it are to 
be controlled, they must he kept simple enough for a single man’s intelligence to 
grasp completely that part for which he is responsible. 

The objective, as shown in figure 5, is to, first, develop a software system 
design of minimTom complexity and, second, to manage the remaining complexity to 
achieve system quality and control costs and schedules. This point is further 
illustrated in figure 6, There will exist a bounded design space within which the 
system requirements are satisfied. Within this system space, the least complex 
design will be the better design. Some important factors in achieving this ob- 
jective are discussed in the following paragraphs. 
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MANAGEMENT ATTENTION 


Figure 7 is a coramon diagram, but appropriate. The ability of managers and 
technical developers to influence the outcome of a project is very high at the 
outset but declines rapidly as commxtments are made, schedules and budgets are 
used up, and code and documents are produced. If ideas, decisions, commitments, 
etc., are not well defined and communicated, management attention will be required 
as shown by the curve dominant on the right side of figure 7. Disproportionate 
attention will be required at acceptance when the disparities between requirements 
and system capabilities are found and again during use when the system is applied 
to problems other than those demonstrated at acceptance. As shown, these are the 
very times when little ability remains to influence the outcome of the system ex- 
cept by rebuilding the deficient parts . 

The goal is to give management attention as shown by the middle curve on the 
left hand side of figure 7. This curve follows the same form as the curve repre- 
senting ability to influence outcome (l.e. , management attention is given to the 
system development at the time of greatest effectiveness) . 

It is important to note that a marked distinction should be made between 
management, of which there is often too little, and manager presence, of which 
there is often too much. Management is that work involved in planning, organizing 
leading, and controlling the project, regardless of who does it. 

INFORMATION REQUIREMENTS 

The general information requirements are shown in figure 8. These require- 
ments are shown as isolated islands of "things, knowledge and data" pertaining to: 

Problem characteristics 
External system characteristics 
Internal system characteristics 



Generally! a great deal is known about each of these, but this knowledge is, 
in the general case, diffused among many individuals and organizations. More 
importantly, it is not organized relative to the new system being developed. 

To amplify, new systems are usually developed as a further increment in 
capability and productivity in an existing environment. Users in the problem 
solving environment possess a great deal of information essential to the success 
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of a new system. Similarly, those who have advanced the engineering technology 
will have developed' new external system characteristics compatible with that 
advancement. Further, new advances will have been made in computing hardware and 
operating systems, software languages, control logic, numerical methods, etc. that 
will have an important effect on the internal system characteristics. The task in 
developing a new system is the organization of this information. 


TOP DOWN SYSTEM DEVELOPMENTS 

The organization of information cannot be achieved until two conditions exist: 

1. Communication is established between the isolated islands. 

2. A disciplined procedure is imposed for organizing and iterating 
between and within the isolated islands. 

The approach is that shown in figure 9 . 

1. A problem analysis is performed to establish a model for the 
process or problem the new system is to support or solve. 

2. A requirements definition is made based on the problem 
characteristics that were organized as a result of the 
problem analysis. This requirements definition establishes 
usage procedures, technology, and relationships that form the 
external system characteristics, or, more conventionally, the 
engineering specifications . 
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3. The external system requirements thus organized form the 
foundation for performing the various levels of system 
requirements. The system can then he implemented. 

This bridging and organizing process is illustrated in figure 9. The bridges 
shown are two way. They will be much traveled while the system characteristics 
are iterated to achieve an optimum implementation. This process of organizing 
and iterating prior to implementation is essential and should consume about fifty 
percent of the project budget and schediile. A discipline is required, however, 
for organizing and iterating information in a consistent manner that will converge 
to the objective stated in figure 5. This discipline Is the structured approach. 

THE STRUCTURED ARPROACH 

There are three basic elements in the structured approach (figure 10) : 

1. A hierarchal grouping, or tree structure, is formed by 
decomposing general statements into levels of more detailed 
statements. This grouping must be complete (t.e., all functions 
or processes to the lowest level must be shown) and it must be 
self consistent (l.e., the functions or processes shown must be 
implementable) . When complete, this grouping represents the 
static relationships of data, 

2 . Each node in the hierarchal grouping has the characteristics 
defined by the basic node. In the Illustration of figure 10, 
the basic node is a process. The process is fed by known 
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control, data, and devices and produces a known output. 

Other basic node characteristics may be defined as reqioired. 

3. The nodal relationships are established by analysing control 
and data communication between the nodes at each level in 
the hierarchal grouping. When complete, this analysis es- 
tablishes the djnamic relationships of the system. 

The process begins with a very general but complete statement of the output 
required of the new system. The process by which the output can be obtained is 
then defined. Necessary control data and device information, again general but 
complete, are then determined. The process in the top level node is then decom- 
posed into its component parts and each of these is analyzed in precisely the same 
manner as top level node. In addition, the dynamic relationships between nodes 
are analyzed to determine control structures, data structures, and device require- 
ments. Decomposition is continued downward until all primitives necessary for 
implementation of software are defined and designed. 

This structured approach then forms the basis for organizing all of the 
work and products associated with the system development (i.e., management of 
the system development becomes possible and takes on a meaningful role) . 

SYSTEM MANAGEMENT 

The work of management, as shown in figure 11, is to plan, organize, lead, 
and control. Through the structured approach, a work breakdown structure can be 
established for each stage of the system development. 

The work breakdown structure forms the basis for estimating schedules and 
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budgets and for making work assignments. Since, in the structured approach, the 
design process and its documentation are always highly ■'visible, control can be 
exercised over budgets, schedules, and quality through meaningful reviews as the 
system is decomposed and its components designed. Since the development proceeds 
in an orderly conrrolled manner always within the intellectual grasp of the developers, 
the system always remains manageable. The complexity that results from plunging 
immediately into the details of the system has been replaced by a controlled step- 
wise procedure in which each step is no more complex than the previous one. 

In conclusion, this process is not new. It is used routinely under other 
titles for the organization of complex Information and for the development of 
essentially all fabricated products from lawnmowers to airplanes to satellites . 

Everyone who has used a dictionary, obtained a part from a Sears store, or used 
engineering drawings has used the structured approach. Its application to software 
development is recognition that software is a fabricated product. It has been diffi- 
cult to think of software as a fabricated product since the medium of expression is 
not metal or glass or plastic. In fact, there is no parallel to these in the 
software product. The medium of expression is logic and language and mathematics. 

The formation of the product consists wholly of organizing this medium of trans- 
forming data successively through a series of processes until the output require- 
ments are met. 

The use of the structured approach makes possible a fabrication of the software 
product in which efficiency, modularization, extendability , reliability, and main- 
tenance can be meaningfully designed into the system. Thank you. 
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Harvey McComb: 1 

Next speaker is Harry White, from the National Bureau of Standards, to discuss 
Standards and Documentation. 


Harry White, National Bureau of Standards: 

I start off first of all conversing. I would like to take an exception to the 
previous speaker. And sincerely the point- is that software is not the dully mess 
that we found it to be several years ago. But there are still many professional 
aspects which we have not been able to determine, standardize, or handle that we need 
to consider in order to adequately address essential software as a tangible resource. 

The last couple of years we have been able to more accurately identify the cost 
of software, which is one of the most essential aspects of computer systems. We 
have fotind that software is the most expensive element in computer systems, software 
and data generation. Unfortunately, from a management standpoint, we concentrated 
in the early days on the aspect of the hardware. We can count the number of computers 
and the number of peripheral equipment and the number of tapes that are in the library. 
But this is not where the real problem is in management of computer systems . 

As far as users are concerned, they care less about the hardware as long as 
they effectively get the job done. And looking at the investment made over the years, 

1 

it has become clear that the cost of software and cost of maintenance have been grossly 
underestimated. Therefore, in the last 15 years we have been concentrating upon es~ 
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tablishing standards for software development and maintenance. Essentially, that is 
one of the highlights of the presentation I want to give this afternoon. First of 
all, there is a little bit of bachgronnd that I think you might be interested in. 

What is the Bureau of Standards doing in the computer area, and particularly what 
do our standards mean? Essentially, our charter for the work at the Bureau of 
Standards is very recent, recent in terms of 10 years, and it is based upon the 
legislation enacted back in 1965, referred to as the Brook Act, which placed with 
the Secretary of Commerce the responsibility for establishing standards for the 
effective uses of computer services within the federal government. To this end, we 
work very closely with industry on a voluntary basis, manufacturers, the American 
Standards Institute, and International Standards Organization on an international 
basis in arriving at the standards in computer area. 

Furthermore, we have a number of federal test groups and public advisory com^ 
mittees that we work with on a day to day basis in arriving and identifying the 
standard parts in the Federal ADP communities. 

The highlights of the presentation today are the role of documentation and 
standards as it pertains to the aspects of determining compliance with software, the 
capability of being able to share software and, furthermore, the very hairy question 
of how do we certify software and what does that mean. 

The position of the National Bureau of Standards, and this is arrived at over 
many years of studies. Is that the key element to assuring functional fidelity for 
computer based Information systems is precise documentation. Documentation serves 
as the standard for testing* a system against expected design and performance specifi- 
cations . 

The testing of computer programming language compilers for compliance is 
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always performed using a standard (documentation) as the basis. All deviations 
are reported in terms of their departure from the standard. 

Documentation, standards and the capability to precisely describe systems 
requirements intis t be recognized, demanded, and practiced before computer techno- 
logy truly becomes a profession rather than an art. Without documentation 
standards and practices, it is impossible to validate or certify the performance 
of a computer system. 

Now let us talk about standardization in the computer field and how this 
relates to users. Essentially, standardization in the computer field is depen- 
dent upon user's demand for standards; we cannot assume, based upon our current 
market place practices, that the vendor, whether it is a hardware or software 
vendor, is going to provide you with product standards. Our experience cur- 
rently in the market place is that corporate standards are the market place 
practice, and if you as users want standards, you are the one that has to demand 
them. IBM and Sperry are not going to give you voluntary standards. There 
is a very good marketplace reason why this is being practiced today since 
this type of situation is being handled more or less on a legal basis, and 
is a subject of antitrust. 

Another problem that we experience essentially is that documentation standards 
have not become part of the professional standards. In the ADP (Automatic Data 
Processing) field, since it is essentially a new evolving technology, we have not 
progressed to the point that we have in other professions of recognizing the im- 
portance of standards as part of the professional practices. 

And here I challenge you as professionals in your field to consider the aspects 
and the importance of standards, documentation practices as they pertain to the im^ 
portance of your profession. And finally professional standards and practices must 
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be iiser defined and implemented r It 3, not the role of vendors to define the 
type of standards that you need; this is your problem and the solution rests with 
you. 

The National Bureau of Standards has the responsibility for the development of 
standards to provide for the effective acquisition and use of computer systems within 
the fede!ral government. This responsibility is derived from the provisions of Public 
Law 89-306 which was enacted in 1965. One of the highest priorities for standards 
is for documentation that will facilitate the specification, operation, maintenance, 
and sharing of coi^uter software. Software costs now far exceed the costs for hard- 
ware. Unfortunately, we have not yet learned how to effectively account for and 
manage software as we have hardware. NBS has a major program that is focused on 
software management. We have developed and published five federal standards and 
guidelines that are currently being implemented by federal computer activities.^ 
These standards and guidelir?s were developed based upon active participation and 
inputs from government, industry, and using ADP (Automatic Data Processing) organi- 
zations. A brief description of these standards is provided below: 

Vocabulary for Information Processing , Federal Information Processing Standards 
Publication 11 (FIPS PUB 11) . This publication provides an alphabetic listing of 
over 1200 terms and definitions used in information processing. 

Guidelines for Describing Information Interchange Formats (FIPS PUB 20) . Also 


^Information concerning the availability of these Federal Information 
Processing Standards may be obtained from the Office of ADP Standards 
Management, National Bureau of Standards, Washington, D. C. 20234. 

(Phone 301-921-3157.) 
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American. National Standard XlO.1-1973 . These p-ublications provide documentation 
guidelines for identifying and describing the physical and logical characteristics 
of formatted information involved in automated interchange. 

Flowchart Symbols and Their Usage in Information Processing (flPS PUB 24) . This 
publication provides standard flowchart S3nnbols and specifies their use in the 
preparation of flowchart documentation used to describe computer systems and pro- 
grams. 

Software Summary for Describing Computer Programs and Automated Data Systems (FIPS 
PUB 30) . This publication provides a standard software summary form (Standard Form 
185) and instructions for describing computer programs and automated data systems 
for purposes of identification, reference, and dissemination. This standard is used 
for documenting software that is developed or acquired for federal government use. 
This standard is also used by the General Services Administration as a basis for 
registering and identifying software in the Federal Software Exchange Program. 
Guidelines for Documentation of Computer Programs and Automated Data Systems (FIPS 
PUB 38) . These guidelines provide a basis for determining the content and extent of 
documentation needed for effectively describing computer programs and automated data 
systems. Ten document types are provided for the various aspects of systems design, 
operation, and maintenance. These include; ’ 

1. Functional Requirements Docmnent . Provides a basis for the mutual 
tmderstanding between users and designers of the initial definition 
of the software, including the requirements, operating environment, 
and development plan 

2. Data Requirements Document . Provides, during the definition state of 
software development, a data description and technical information 
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3. 


about data collection requirements 

System/Subsystem Specification . Specifies for analysts and program- 
mers the requirements, operating environment, design characteristics, 
and program specifications for a system or subsystem 

4. Program Specification . Specifies for programmers the requirements, 
operating environment, and design characteristics of a computer program 

5. Data Ease Specification . Specifies the identification, logical charac- 
teristics, and physical characteristics of a particular data base 

6. Users Manual . Describes the ftonctions performed by the software in 
non-ADP terminology such that the user organization can determine its 
applicability and when and how to use it. It should serve as a reference 
document for preparation of input data and interpretation of results 

7. Operations Manual . Provides computer operations personnel with a 
description of the software and of the operational environment so that 
the software can be run 

8. Program Maintenance Manual . Provides the maintenance programmer with 
information necessary to imderstand the programs, their operating 
environment, and their maintenance procedure^ 

9. Test Plan . Provides a plan for the testing of software; detailed 
specifications, descriptions, and procedures for all tests; and test 
data reduction and evaluation criteria 

10. Test Analysis Report . Documents the test analysis results and findings; 
presents the demonstrated capabilities and deficiencies for review; and 
provides a basis for preparing a statement of software readiness for im- 
p 1 ement at ion 


25 




In summary, there are several key observations concerning documentation and 
standards that should he noted by users and managers of computer systems and 
services: 

1. Documentation is the key element for the eventual standardization, 
validation, and maintenance of computer software. Documentation 
standards and practices must be included as an integral part of 
systems management, development, and operation. 

2. Currently, the computer profession does not adhere to or utilize 
accepted documentation procedures as do other professions. Standards 
have been developed and are now available. The responsibility for stan- 
dards implementation now rests with individual systems managers and 
computer professionals. 

3. Currently, users, and managers must explicitly cite the use of stan- 
dards when acquiring or developing computer systems. Unlike other 
industries, the computer marketplace is currently dependent upon the 
individual corporate practices of the major vendors . Unless the users 
and acquirers of computer services and systems demand standards , this 
will continue to be the marketplace environment which inhibits compat- 
ibility and sharing of computer hardware, software, and data across 
different product lines . 

In conclusion, I hope I have been able to whet your appetite for some of these 
standards. I have provided you with information about where they can be acquired. 

As I mentioned, they are available from the National Bureau of Standards and we also 
have a complete set of all our standards that has been published to date, which is 
39 standards included in a specialized three way binder that holds these. Thank you 
very much. 
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Harvey McComb; 

Next is Bob Nickell, from Sandia Corporation, and his topic is Software 
Validation and Certification. 


Bob Nickell, Sandia Corporation; 

I guess I feel like we all have been here before or at some other similar 
place. It is more like reincarnation, we seem to resurrect this animal periodically 
every couple of years or so; we examine it to find out whether or not it has ob- 
tained the proper purity that we placed alongside the godS; and then we dispense with 
it and resurrect it again in two years . 

The two topics that I am supposed to address today (very briefly by the way and 
I will save most of the session for the discussion between the audience and 
the panel members later on this afternoon) are two very different concepts. The 
idea of validation is a operational requirement on software whereas the concept of 
certification is a quasi-legal or even possibly legal requirement for software and 
quite possibly for people that use software substantially. And in the past few years 
while we have all been talking about the subject, now very quietly down at the grass 
roots level, a software vendor and software users are reaching a marketplace accom- 
modation on just which of these concepts is important, what kind of certification 
seems to be in order, and so forth. 
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Although, large software systems have heen developed for a variety of engi- 
neering disciplines, the title for this symposium suggests that the emphasis 
shotild be upon the validation and certification exercises that have been under- 
taken relative to structural analysis. This discussion, therefore, will attempt 
to describe the terms validation and certification in their structural analysis 
context, point out some current efforts in both regards, considering both advan- 
tages and limitations, and speculate on the trends of the future. 

Two of the components of the validation process - verification and qualifi- 
cation - have been defined by Griffin (ref. 1) and are shown in figures 1 and 2. 
Verification, which is primarily the responsibility of the software developer, 
involves the demonstration that the theories and models upon which the program is 
based are correctly coded, irrespective of their applicability to design. Examples 
might be a particular constitutive model (say, a dilatational-dependent plasticity 
theory) or a particular finite element description (say, an arbitrary doubly- 
curved deep shell element) . Qualification, which is primarily the responsibility 
of the software user, involves the demonstration that the program capabilities will 
not be exceeded by combinations of elements, boundary conditions , Initial conditions, 
material models, etc., that are typical of the user's design analysis requirements. 
This exercise should also determine that the logical flow paths and program config- 
urations (core size, backing storage, dimensions, and other critical parameters) 
are consistent with analytical needs. The developer also maintains a set of quali- 
fication problems which serve two purposes: (1) to requalify the program after each 
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set of updates or increment of capability and (2) to provide a mechanism for work- 
shop training of prospective users . 

An additional component of validation that has become prominent in recent 
years is calibration, (see fig. 3) the pro^-edure by which simplified models of 
complex physical events (sometimes involving experimental correlations) are incor- 
porated into the software, often through free parameters. An example might be the 
calibration of an uncertain mass distribution through a free vibration survey, 
followed by transient design analysis. 

These three components of validation have been supplemented by occasional 
thoughts on user qualification (refs. 2 and 3), which deals with the education 
and training of analysts in such areas as numerical integration, solution 
algorithms, linear algebra, .and the like. 

Certification, on che other hand, is not a technical concept but, instead, 
implies legal standing for the software package (program certification) or for the 
user (professional certification) . The interest of somebody with an ability to 
provide legal sanctions against uncertified software or users is also required. A 
somewhat weaker form of certification would accompany an ability to impose economic, 
or perhaps moral, sanctions. For example, most governmental agencies with 
large research and development budgets (such as those shown in fig. 4) would 
be able to exert considerable influence over software and users of software 
under contract to them although no formal attempts at certification by such 
agencies have come to ,my attention. Instead, validation has received major 
emphasis, as can be seen by examining the ERDA-sponsored (ORML) High- 
Temperature Structural Design Methods Program and Validation of High- 
Temperature Design Methods and Criteria Program (figs. 5-9). 
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Other official bodies do have implied legal standing with regard to software, 
should they choose to become involved. TVo of these bodies are listed in figure 10. 
Neither has chosen, to venture into the treacherous waters of certification al- 
though the NRG position on benchmark calculations often wavers precariously between 
software qualification (for certain types of safety analysis) and software certi- 
fication. The ASME Boiler and Pressure Vessel Code is not apt to be extended to 
treat certification either, with the possible exception of postprocessors (i.e., 
software that converts stress analysis output into a form that can be compared to 
Code allowables) . A likely course of action would be to address postprocessing 
procedures but to avoid explicit consideration of the postprocessors themselves. 

Professional societies have a vested interest in promoting certification of 
individuals, both through the legal formalism of professional registration and the 
quasi-legal accreditation of professional schools or institutions of higher learning. 
A recent development is the adoption of the concept of xecertificatlon through 
continuing education by the professional societies - a concept that might eventually 
lead to a form of software user certification. 

A more likely candidate description for people certification was offered by 
Don Griffin at the Winter Annual Meeting at the American Society Mechanical Engi- 
neering of 1972, He viewed people certification from a viewpoint similar to the 
AMA view. At present we require registration which is our legal certification for 
determining qualification to design structures, buildings, and engineering parts. 

His view was that registration may be (he did not say it was) but it may be insuf- 
ficient to protect the public safety. In other words, if it becomes widely accepted 
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that the registration, exam and the registration certification are no longer suffi- 
cient to guarantee the public safety (as it is not in the medical field) then we 
must have another level of certification. In the case of AMA, they specify the 
certification requirement and also require an oral exam and written examination 
in a submedical specialty. 

Now it may be that in the software area such sublevel certification is a 
requirement because most registered engineers are assigned complex tasks these 
days and need to use complex algorithms, probably without sufficient understand- 
ing of the algorithm and its limitations, even though it is used to perform 
complicated stress analysis. If in fact that turns out to be the case, we can 
make an argument, a serious argument, for some type of certification of software 
users. 

One last concept I woiild like to discuss, the Nuclear Regulatory Commission, 
does in fact have a certification or a quasi-certification document on the accep- 
tability of computer programs for the design analysis of mechanical systems, and 
it put these programs in a precategory and I would like to just read to you 
three short sentences. Either of the programs must fall into these categories; 

1. The computer program is a recognized program in the public domain 

and has had sufficient history of use of justified complexity covering 
the entire domain. 

2, The computer program’s solution to a series of test problems with 
accepted results has been demonstrated to be substantially identical 
to those obtained by similar independently written programs in the 
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public domain. In other words, you may wish to use a 
proprietary piece o£ software but you must demonstrate 
that it has the same general procedure, plus the same 
solution, as a program in a public domain. 

3. The program solution to a series of test problems is 

substantially identical to those obtained by hand calcu- 
lation or accepted experiments or analysis results 
published in the Technical Literature. Thus, you have a 
choice. 

The situation, both today and in the future, could be summarized 
as follows: (a) the verification process Is well understood since it 

involves a one-to-one correspondence between concepts of structural 
mechanics and the programmer’s ability to replicate these concepts; 

(b) the q^ualification process is not well iinderstood since the inter- 
action between program logic, solution algorithms, and user-controlled 
factors (e.g., program configuration, space and time discretization, 
etc.) does not lend itself to formal academic presentation; those 
engineers with expertise in program qualification tend to acquire such 
skill piecemeal, over a period of years; and (c) the certification of 
software, in addition to its troublesome legal implications, does not 
appear to offer any relief to the natural tension between the owner of 
a structure, the designer/analyst, and the software developer. Thank 
you. 


32 



References 


(1) Griffin, D. S.: The Verification and Acceptance of Computer 

Programs for Design Analysis. On General Purpose Finite 
Element Computer Programs, P, V. Marcal, ed., American Soc. 

Mech. Eng., c.1970, pp . 143-150. 

(2) Nickell, R. E.: Qualification — People, as Well as Programs . 

Engineering Computer Software — Verification, Qualification, 
Certification, I. Berman, ed., American Soc. Mech. Eng., 

C.1971, pp. 51-59. 

(3) Nickell, R. E.: U^er Education: The Finite Element Short Course. 

The Software User; Education and Qualification, H. Kraus, ed., 
American Soc. Mech. Eng., c.l972, pp. 9-15. 


33 



V E R I F I C AT I 0 M 

VERIFICATION IS DEFINED AS THE DEFDNSTRATION THAT A 
COMPUTER PROGRAM CORRECTLY SOLVES THE MODEL THAT 
WAS PROGRAMMED/ INDEPENDENTLY OF WHETHER OR NOT 
IKE MODEL IS A VALID REPRESENTATION OF ANY PARTICULAR 
SYSTEM. 

Figure 1 
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QUALIFICATION 


QUALIFICATION IS CONCERNED WITH THE USE OF COMPUTER 

PROGRAMS FOR SOLVING THE REAL PROBLEMS ENCOLW7ERED IN 

DESIGN, GIVEN THAT A COMPUTER PROGRAM CORRECTLY SOLVES 

THE MODEL PROGRAFTED/ DOES TWE COMBINATION OF 

MATHEMATICAL MODEL/ DISCRETIZATION/ DESCRIPTION OF 

MATERIAL PROPERTIES/ REPRESENTATION OF THE LOADING AND 

TEMPERATURE HISTORIES/ AND BOUNDARY CONDITIONS, ALL 

CONSISTENT WITH THE PROGRAM LIMITATIONS, GIVE AN 

ACCEPTABLE SOLUTION TO THE PHYSICAL PROBLEM ? 

Figure 2 
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COMPUTER PROGRAM VALIDATION 


VERIFICATION - DEVELOPERS' RESPONSIBILITY 

QUALIFICATION - CO^BINED DEVELOPER AND USER 

RESPONSIBILITY 

(calibration) 

Figure 3 


ECONOMIC AUTHORITY 

DEPARTMENT OF DEFENSE 

ENERGY RESEARCH AND DEVELOPI^NT ADMINISTRATION 

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 

Figure 4 
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Table 1. Typical Set of 


Problem No. Description 

1.1 Uniaxial specimen subjected to 
reversed cyclic loading into 
plastic range 

1 . 2 Uniaxial relaxation specimen 
under isothermal conditions 


1.3 Uniaxial specimen subjected to 
stepped and reversed creep 
loading 

1.4 Thick -walled cylinder subjected 
to transient radial temperature 
gradient producing plastic 
behavior 

1.5 Biaxially loaded plate or 
cylindrical shell subjected to 
specialized nonproportional 
loadings into plastic range 


Figure 5 
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Benchmark Problems 


Comments 


Analysis should produce cyclic 
stress -strain curve. Good for 
studying load incrementing 
choices . 

Analysis should reproduce simple 
calculations or actual relaxa- 
tion test results. Good for 
studying time increment choices 
in creep analyses . 

Analysis should produce simple 
creep curves with hardening 
rules considered. Good for 
time-increment selection. 

Results can be checked 
analytically. Good for studying 
choice of temperature increments 
for plastic loading. 

Results can be checked 
analytically or, in the case of 
cylindrical shells subjected to 
pressure, axial, and torsional 
loads, by experimental results. 
Good for checking program treat- 
ment of nonproportional loading. 
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Table II. Typical Set of Qualification Benchmark Problems 


Problem No. 


Description 


Comments 


2.1 


2.2 


2.3 


2.4 


Simply-supported beams 
subjected to various center 
loading histories producing 
elastic-plastic creep behavior 

Simply-supported circular 
plates subjected to various 
center loading histories 
producing elastic-plastic 
creep behavior 

Capped circular cylindrical 
shells subjected to varying 
internal pressure loading 
producing elastic-plastic 
creep behavior 

Finite-width flat plate with 
circular hole subjected to 
cyclic axial load producing 
elastic-plastic creep behavior 


For plane 2D-solid and beam 
analyses. Test results being 
obtained under ORNL High- 
Temperature Structural Design 
Methods Program. 

For axisyrametric 2D-solid and 
flat plate analyses. Test 
results being obtained under 
ORNL High-Temperature Structural 
Design Methods Program. 

For axisyiranetric 2D-solid and 
shell analyses. Test results 
being obtained under ORNL High- 
Temperature Structural Design 
Methods Program. 

For plane 2D-solid analyses . 
Results being obtained under 
ORNL High-Temperature Structural 
Design Methods Program. 


2.5 Stiffened shear lag panel 

subjected ,to cyclic external 
loading producing elastic- 
plastic creep behavior 


For plane 2D-solid analyses. 
Results being obtained under 
ORNL High-Temperature Structural 
Design Methods Program. 
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(Table II cent.) 


Problem No. 


2.6 Pipe thermal ratchetting 
specimens (straight pipe 
subjected to internal pressure 
and to periodic thermal down- 
shocks in contained fluids) 

2.7 Piping elbows subjected to 
external loadings producing 
elastic-plastic creep behavior 


2.8 Nozzle attachment thermal 
ratchetting specimen 
(conditions as above) 

2.9 Nozzle-to-spherical shell 
attachments subjected to 
internal pressure and 
external loadings causing 
elastic-plastic creep 
behavior 

2.10 Intersecting cylindrical 
shells or piping tees 
subjected to external 
loadings producing elastic- 
plastic creep behavior 


For axisymmetric 2D-solid or shell 
analyses. Results being obtained 
under ORNL High-Temperature 
Structural Design Methods Program 
and by LMEC. 

For shell and special piping 
analyses . Test results being 
obtained under ORNL High -Temperature 
Structural Design Methods Program 
and Validation of High-Temperature 
Design Methods and Criteria Program. 

For shell analyses . Results being 
obtained under Validation of High- 
Temperature Design Methods and 
Criteria Program. 

For axisymmetric 2D-solid or shell 
analyses. Results being obtained 
under ORNL High-Temperature 
Structural Design Methods Program. 


For shell analyses. Test results 
being obtained under ORNL High- 
Temperature Structural Design 
Methods Program and Validation of 
High-Temperature Design Methods 
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(4) Thick-Walled Cylinder Subjected To 

Radial Temperature Gradient Producing 
Plastic Behavior 



(5) Flat Plate Or Cylindrical Shell 

Subjected To Special Nonproportional 
Biaxial Loadings Prodi’cing Plastic 
Behavior 

Figure 8. Simple Benchmark Problems for User Training and Basic 
Program Verification. 
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Simply-Supported Beams 





(2) Simply-Supported Flat 
Circular Plates 
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(3) Capped Circular- 
Cylindrical Shells 



(4) Finite-Width Flat Plate 



No 




(6) Pipe Thermal Ratchet ting 


(Forces) 
— ^(Moments) 


(7) Piping Elbows 



(8) Nozzle Attachment Thermal Ratch- 
et ting (With Internal Pressure) 



(9) Nozzle-To-Sphere Attachments 



(5) Stiffened Shear Lag Panel (10) Intersecting Cylindrical 

Shells (With Internal 
Pressure) 


Figure 9. Advanced Benchmark Problems Qualification of Program 
and User. 


LEGAL AUTHORITY 

ASME -BOILER AND PRESSURE VESSEL CODE 

U, S, NUCLEAR REGULATORY COMMISSION 
Figure 10 
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Harvey McComb; 

Next is Jim Johnson, from AFFDL, Dissemination, Portability, and 
Maintenance. 

Jim Johnson, AFFDL: 

Yesterday I managed to get soaking wet; it rained in Washington, yet I 

had a raincoat in the Motel where I was staying. Sure enough I was unable to 

maintain all in ,good health; consequently my throat is a little raw and I may 
lose my voice before 5:00 p.m. Also, the Department of Defense is suffering 
from an economic recession. Consequently we are a little unable to get you 
visual aids for this presentation. While that was going on we had a reorganiza- 
tion in the laboratory, and I got a physical relocation. Perhaps I should 
avoid any such remarks and focus primarily on Dissemination, Portability, and 
Maintenance of Large Engineering Software Systems. 

My discussion does not address the many subtle problems of our panel theme. 

Nor will I attempt to define a workable solution to every aspect of our panel 

theme. Instead, I shall focus primarily on Dissemination, Portability, and Main- 
tenance of Large Engineering Software Systems. 

The complexities and problems in connection with the development , distri- 
bution, and maintenance of large engineering software systems are widespread. 
Within the past decade, most of us have been bombarded with large computer 
programs such as ASKA, COSMOS, ASTRAL, SAS, STAIR, STAEIDYNE, FORMAT, MAGIC, ASOP, 
NASTRAN, FLEXSTAB, etc. for the solution of our engineering problems. Upon 
receipt of one of the large general purpose structural analysis computing systems. 
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we attempt to load the program aad execute a check case. At this point, we usually 
face the harsh realities of life. In most instances, it is not possible to success- 
fully load the program on the first try. Other abortions result from tape/card 
processing problems, inadequate definition of minimum machine resources required, 
etc. Tet we continue to obtain other people's software in spite of the many frus- 
trations to be expected because of our need to utilize the latest technological 
developments of other engineering organizations. Thus, the most important aspect of 
dissemination, portability, and maintenance is technology transfer. To date, I 
believe that too much emphasis is placed on the costs of doing this job rather than 
the objective of technology transfer. Since the large general purpose programs are 
developed to serve the scientific and engineering community, it is only necessary to 
maintain a proper balance between the needs of the practitioners and the costs of 
computer program maintenance and distribution. Enough for costs. Let us examine 
software design and development procedures to facilitate maintenance and those 
procedures and organizations to facilitate dissemination in the light of our ob- 
jective to effect good technology transfer. 

For large engineering software systems, good technology transfer is accom- 
plished via: 

1. Documentation 

2. Users' Meetings 

3. Education and Application Seminars 

Inasmuch as documentation and user education will be discussed in detail by 
other panelists, I shall focus primarily on those procedures and practices that 
must he considered as an integral part of the aforementioned areas to effect 
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technology transfer. 

When a computer program cannot he understood by a potential user, the 
^ould-be recipient may write his ovn program, which results in an unnecessary 
duplication of effort, or the problem to which the program would have been 
applied will remain unsolved. First, and foremost then, the dissemination and 
portability of large computer programs require "good communication". Although 
good criteria for the dissemination of large computer programs have been around 
for more than a decade, for completeness I shall restate a lis of requirements 
that insure good communication when disseminating large general purpose computer 
programs : 

1. Complete statement of the program, its problem formulation, and com- 
plete equipment description 

2. Functional flow charts 

3. Fully documented source program listings 

4. A complete user’s manual containing a clear description of the inputs 
and outputs, operating instructions, and symbol definitions 

5. Adequate test cases 

6. All assembly (machine) language shall be easily identifiable and 
emphasis will be placed on localizing assembly language coding 

7. Master code media 

Most of us are quite familiar with all of the above items. We at WPAFB have 
contractually required the satisfaction of this list since about 1962 or 1963. 

In spite of our familiarity, let us amplify a few of the items. 

Functional flow charts: A complete description of each function must he in- 

cluded. This definition includes the title, purpose and use, system inputs, ex- 
pected output and results, relationship to other functions, and summary of function 
operation . 
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User manual; A complete description o£ the operating instructions and phi- 
losophy. The operating instructions must include step-by-step procedures required 
to initiate (or implement) the program, maintain computer program operation, ter- 
minate and restart the program (normal and unscheduled termination), software 
system modification/generation ("octal-in" and creation of a new version of the 
master code media), symbolic updating procedures, and minimum machine resources 
required. 

Assembly (machine) language: Programs containing machine language code are 

strongly dependent upon the characteristics of the computer itself, and programs 
containing assembly language code are generailly not pprtable -to any machine other 
than the one for which they were written. 

Master code media;. The usual mode for dissemination of large programs is by 
magnetic tape. The master tape description is a must, and this description must 
define the format, density, blocking, mode, contents, and processing' procedures , The 
normal (or recommended) procedure for dissemination of master code is to copy the 
source program and test cases onto a magnetic tape, then compile the program from the 
copy, and exercise this compiled program on the test cases fr'jm the copy. This pro- 
cedure insures that no malfunction occurred during copying and, since this output is' 
also sent to the recipient, it aids him in putting the program up on his system. 

Now let us turn our attention to maintenance procedures. Maintenance, at most, 
implies the requirement to "stay current". Technology is dynamic and maintenance of 
large engineering software systems necessitates a "constant process" of modification/ 
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updating to provide for new capability and error fixes. Is this new information 
automatically distributed? Who makes the necessary modifications? When? Why? 

Is there feedback from users? These are the questions posed by the engineering 
community, however, program maintenance involves these simple procedures: 

Program code revision/modification; Anyone can perform this task. It re- 
quires some knowledge of the program to be changed. Changes (enhancements and 
errors) must be documented (symptoms and conditions) , must work and not adversely 
effect the rest of the program, must be verified and validated, and must be disse- 
minated. 

Reference program version: Must be clearly identified and used for maintenance 

work. 

Release of new version: 24 to 48 months during which time a significant 

number (possibly 50) of code modifications have been made. 

Up-to-date user identity: Addresses and identities of users (old and new) are 

required to maintain currency. 

Dissemination: Changes, modifications, error fixes, and the like have no real 

significance unless they are available and distributed in a timely manner. 

The foregoing remarks outlined software design and development procedures to 
facilitate dissemination, portability, and maintenance. Let us briefly turn our 
attention to procedures and organizations to facilitate dissemination. 

Large engineering software systems are disseminated in one of two ways. 

Firstly, the program is distributed to a customer (requestor) via master code 
media. This procedure requires customer (local) implementation and maintenance. 
Secondly, the program is available to the customer via some network system. This 
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procedure requires no custonjer (local) Implementation and maintenance. Programs 
available via network systems only require dial-up procedures. For this reason, the 
following information pertains to procedures of organizations for direct distribution 
of large programs via master code media. 

1. Programs are stored in a central depository, 

2. Information defining the availability of these programs is 
published via con^iuter program abstracts, newsletters, and 
technical reports. 

3. The costs for obtaining a program from one of the depositories 
vary from zero, to exchange programs, to annual leases (or 
purchases), up to $40,000.00. 

4. Several program distribution centers have defined, or are in the 
process of defining, criteria for program inclusion in the depo- 
sitories . 

5. Checkout procedures for requested programs consist of compilation 
and execution of test cases, at most. 

6. Available documentation is supplied. Implementation instructions 
vary from no information to "customizing”. 

7. Very limited maintenance, if any, is attempted. 

The Air Force Flight Dynamics Laboratory at Wright-Patterson Air Force Base 
has distributed engineering software since the early sixties. The Aerospace 
Structures Information and Analysis Center (ASIAC) was established in 1974 to 
collect and disseminate information on aerospace structures. Products and services 
available from ASIAC include: 

1. Technical Inquiry Services on Aerospace Structures Problems 

2. Bibliography Service 

3. Monographs and Reference Books 
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4. Current Awareness Publications. 


5. Computer Program Distribution. 

Approximately 40 structural computer programs are currently avail- 
able from ASIAC. These programs are distributed, if not generally avail- 
at other computer program dissemination centers . In some instances , 

ASIAC has permission to distribute proprietary codes to government agencies 
only. To date, the AFPDL and ASIAC have distributed programs at no charge. 

Our future plans are to recover the handling costs. 

AFFDL and ASIAC perform a significant amount of maintenance and documen- 
tation for the computer codes in our depository. This procedure is the 
notable exception to the generally widespread practices noted above. Thank 
you. 

Harvey Me Comb: 

Next speaker is Professor Steven Penves, from Carnegie-Mellon University, 
representing the Educator's viewpoint. 

Steven Fenves, Carnegie-Mellon University: 

I was not quite sure what aspects of education to cover so I decided to 
put in some personal thoughts. There are three sets of considerations that one 
has to keep in mind in talking about educational aspects of standardization: 
certxfication, maintenance, and dissemination of large engineering software 
systems . 
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First is the distribution of the population to be served by educational 
programs. The vast majority of engineers are program users; their only con- 
tact \vith computer programs is through preparation of Input and interpretation 
of output for programs written by someone else. A much smaller number of 
engineers are ad hoc program developers; they write, modify, or maintain pro- 
grams for their own use, or at most, for use by small groups with which they 
are intimately connected. A miniscule number of engineers are software system 
developers . 

Second, essentially all large engineering software systems are proprietary 
in nature. Many of the aspects of software systems addressed by this panel 
constitute the principal competitive advantage of such systems. 

Third, there is a serious generation gap between engineering software 
system developers on one hand and software engineers on the other. Most of 
the application system developers, myself included, are graduates of the old 
ad hoc school of programming and are not fully aware of recent revolutionary 
developmi its in software engineering such as structured programming, formal 
modularization, and program assembly testing and verification techniques. 

I must call upon a past experience in that category. Exactly 12 years 
ago we made a presentation to IBM of the program we were developing at that 
time, STRESS, and one person asked; "What procedure did you use to resolve 
conflicts at the interfaces?" To me this was an unexpected question, and not 
knowing how to respond, I said: "Well, let's see: there is this group of 

five of us working on the project, my office is in the middle, and I only have 
four chairs, so we borrow a chair to sit down and resolve the conflicts of the 
interfaces." But that was 12 years ago. I think a lot of people have learned, 
something since then. 
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In an attempt to put large-scale software development into some focus, the 
writer has proposed a model of the development process (refs . 1 and 2) . An 
extended version of the model used in a recent study (ref. 3) is shown in figure 1. 

It can be seen that education is shown as a professional activity, reflec- 
ting the above mentioned need to service the largest number of people involved 
(i.e., the end users). Figures 2, 3, and 4 taken from the same survey, in fact, 
classify the available educational activities on the basis of application areas 
covered. Such educational efforts, in general, accomplish their major objective 
of making end users aware of, or even proficient in the use of, existing programs 
without directly exposing the proprietary, cor^etitive aspects of these programs. 

Although comparable survey data are not available, it is fair to state that 
most computer-based courses in formal engineering curricula are geared to the 
second group of engineers (that is, future ad hoc programmers (either graduate 
students who will have to write ad hoc programs to satisfy their engineering 
thesis requirements or students who aspire to use their programming skills to 
secure a competitive advantage in employment)). Needless to say, courses aimed 
at this group are generally taught by people who are themselves ad hoc programmers. 

Education specifically geared to the small number of software system deve- 
lopers is practically nonexistent in the engineering disciplines. From an 
educational productivity standpoint, it is unrealistic to expect a major effort 
in this rather narrow area. 

In Summary, while we all may want more and better educational activities, the 
first two groups of engineers are relatively well served by existing educational 
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activities. In. order to provide the proper education for the third group, two 
things must be done; 

1. An organizational structure must be developed whereby the technical ac- 
tivities shown in figure 1 are open for scrutiny and evaluation without destroying 
the valid proprietary, competitive aspects of the software systems 

2. The educational horizons of software system developers must be considerably 
broadened so that they can apply the relevant aspects of software engineering to 
the development of engineering software. 

Thank you. 
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